Accept async context managers for cleanup contexts (#11681) #11704
Conversation
…\nAdapt async context managers to behave like single-yield async iterators and add tests.
psf-chronographer
bot
added
the
bot:chronographer:provided
CodSpeed Performance ReportMerging #11704 will not alter performanceComparing Summary
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #11704 +/- ##
========================================
Coverage 98.73% 98.73%
========================================
Files 127 127
Lines 43546 43646 +100
Branches 2320 2323 +3
========================================
+ Hits 42996 43095 +99
Misses 390 390
- Partials 160 161 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
aiohttp/web_app.py
Outdated
| # If the callback returned an async context manager (has | ||
| # __aenter__ and __aexit__), wrap it so it behaves like an | ||
| # async iterator that yields once. |
There was a problem hiding this comment.
Would it make more sense to make context managers our de facto default now? Surely then all we need to do for backwards compatibility is:
if not isinstance(ctx, AbstractAsyncContextManager):
ctx = asynccontextmanager(ctx)
There was a problem hiding this comment.
pushed a fix making it the default.
There was a problem hiding this comment.
No, I mean that the code should should be changed to use aenter/aexit instead of anext. Then we don't need the awkward _AsyncCMAsIterator class and much of the other code becomes much simpler.
There was a problem hiding this comment.
- removed the
_AsyncCMAsIterator - replaced the adapter-based approach with direct use of
aenter/aexitfor async context managers.
CleanupContextnow: - If cb returns an
AsyncIterator: advances it once on startup and finishes it on cleanup (legacy). - If cb returns an
AbstractAsyncContextManager: callsaenteron startup andaexiton cleanup. - If cb returns something else: adapts it via
contextlib.asynccontextmanager, callsaenter/aexit.
Dreamsorcerer
added
the
backport-3.14
- fixed doc spelling issues - now uses AbstractAsyncContextManager
for more information, see https://pre-commit.ci
- increased coverage
- reverted spelling_worldlist - moved inside function imports
aiohttp/web_app.py
Outdated
| # If the callback returned an async iterator (legacy async | ||
| # generator), use it directly. If it returned an async | ||
| # context manager instance, enter it and remember the manager | ||
| # for later exit. As a final fallback, convert the callback | ||
| # into an async context manager (covers some edge cases) and | ||
| # enter that. | ||
| if isinstance(ctx, AsyncIterator): | ||
| # Legacy async generator cleanup context: advance it once | ||
| # (equivalent to entering) and remember the iterator for | ||
| # finalization. | ||
| it = cast(AsyncIterator[None], ctx) | ||
| await it.__anext__() | ||
| self._exits.append(it) | ||
| elif isinstance(ctx, contextlib.AbstractAsyncContextManager): | ||
| # If ctx is an async context manager: enter it and | ||
| # remember the manager for later exit. | ||
| cm = ctx | ||
| await cm.__aenter__() | ||
| self._exits.append(cm) | ||
| else: | ||
| # cb may have a broader annotated return type; adapt the | ||
| # callable into an async context manager and enter it. | ||
| cm = contextlib.asynccontextmanager( | ||
| cast(Callable[[Application], AsyncIterator[None]], cb) | ||
| )(app) | ||
| await cm.__aenter__() | ||
| self._exits.append(cm) |
There was a problem hiding this comment.
As I mentioned originally, can't this be reduced to just this?
| # If the callback returned an async iterator (legacy async | |
| # generator), use it directly. If it returned an async | |
| # context manager instance, enter it and remember the manager | |
| # for later exit. As a final fallback, convert the callback | |
| # into an async context manager (covers some edge cases) and | |
| # enter that. | |
| if isinstance(ctx, AsyncIterator): | |
| # Legacy async generator cleanup context: advance it once | |
| # (equivalent to entering) and remember the iterator for | |
| # finalization. | |
| it = cast(AsyncIterator[None], ctx) | |
| await it.__anext__() | |
| self._exits.append(it) | |
| elif isinstance(ctx, contextlib.AbstractAsyncContextManager): | |
| # If ctx is an async context manager: enter it and | |
| # remember the manager for later exit. | |
| cm = ctx | |
| await cm.__aenter__() | |
| self._exits.append(cm) | |
| else: | |
| # cb may have a broader annotated return type; adapt the | |
| # callable into an async context manager and enter it. | |
| cm = contextlib.asynccontextmanager( | |
| cast(Callable[[Application], AsyncIterator[None]], cb) | |
| )(app) | |
| await cm.__aenter__() | |
| self._exits.append(cm) | |
| if not isinstance(ctx, contextlib.AbstractAsyncContextManager): | |
| ctx = contextlib.asynccontentmanager(cb)(app) | |
| await ctx.__aenter__() | |
| self._exits.append(ctx) |
There was a problem hiding this comment.
I tried to reduce it to your suggested version, but I think it changes semantics in a way that breaks cleanup-context lifecycle invariants. We end up with two generator instances because we also call ctx = cb(app) before wrapping, the net effect is that cb(app) is invoked twice in the fallback path:
1- ctx = cb(app) first call
2- contextlib.asynccontextmanager(cb)(app) second call
For async-generator style cleanup callbacks this yields two generator instances. The instance you advance on startup is not the same instance you finalize on cleanup.
I might have missunderstood your comment. Did you intend to change semantics so that cb may be invoked more than once? or did you mean to wrap the returned value when it’s not a context manager?
There was a problem hiding this comment.
the net effect is that
cb(app)is invoked twice
Yes, I had realised that, but not sure it causes any actual problems. We can then probably deprecate the generator fallback in future to remove this, after a couple of years.
The instance you advance on startup is not the same instance you finalize on cleanup.
That shouldn't be true. The second instance replaces the first one before it is used. So, if the first instance is a generator object, it gets discarded without being used at all. The second instance is then used for both startup and cleanup.
There was a problem hiding this comment.
applied your suggestion.
ready for the next review.
…ader_c shim; apply pre-commit automatic fixes
for more information, see https://pre-commit.ci
…epair reader_c shim; apply pre-commit automatic fixes" This reverts commit da51359.
3 participants
What do these changes do?
Adapt async context managers to behave like single-yield async iterators.
1- Allow
app.cleanup_ctxcallbacks to return either:a single-yield async generator (existing behavior), or
an async context manager (e.g.
@contextlib.asynccontextmanager).2- Internally adapt async context managers with a small adapter so:
__aenter__runs at startup (equivalent to the generator yielding), and__aexit__runs at cleanup (equivalent to finishing the generator).3- Preserve previous cleanup/error aggregation semantics.
Are there changes in behavior for the user?
No breaking changes; this is fully backwards compatible.
contextlib.asynccontextmanagerinstances for cleanup contexts.Is it a substantial burden for the maintainers to support this?
No
Related issue number
Closes #11681
Checklist
CONTRIBUTORS.txtCHANGES/foldername it
<issue_or_pr_num>.<type>.rst(e.g.588.bugfix.rst)if you don't have an issue number, change it to the pull request
number after creating the PR
.bugfix: A bug fix for something the maintainers deemed animproper undesired behavior that got corrected to match
pre-agreed expectations.
.feature: A new behavior, public APIs. That sort of stuff..deprecation: A declaration of future API removals and breakingchanges in behavior.
.breaking: When something public is removed in a breaking way.Could be deprecated in an earlier release.
.doc: Notable updates to the documentation structure or buildprocess.
.packaging: Notes for downstreams about unobvious side effectsand tooling. Changes in the test invocation considerations and
runtime assumptions.
.contrib: Stuff that affects the contributor experience. e.g.Running tests, building the docs, setting up the development
environment.
.misc: Changes that are hard to assign to any of the abovecategories.
Make sure to use full sentences with correct case and punctuation,
for example:
Use the past tense or the present tense a non-imperative mood,
referring to what's changed compared to the last released version
of this project.